Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Nonconvex Stochastic Optimization: Nonasymptotic Performance Bounds and Momentum-Based Acceleration

Nonconvex Stochastic Optimization stochastic optimization problems arise in many machine learning problems, including deep learning. The gradient Hamiltonian Monte Carlo (SGHMC) is a variant of gradients with momentum method which controlled and properly scaled Gaussian noise added to the steer iterates toward global minimum. SGHMC has shown empirical success practice for solving nonconvex problems. In “Global convergence optimization: Nonasymptotic performance bounds momentum-based acceleration,” Gao, Gürbüzbalaban, Zhu provide, first time, finite-time context both population risk minimization show that acceleration possible optimization.